Video enhancement algorithms are employed for enhancing the quality, resolution, or frame-rate of video frames in a video sequence or to enhance a still picture output using multiple captured images. These algorithms typically combine spatial and temporal information in an intelligent manner. Noise reduction, exposure correction using multiple images, and super-resolution image generation are some applications of such spatiotemporal combining. Typical spatiotemporal combining methods suffer from either a very high computational complexity or from artifacts due to lack of robustness in the combining process arising from poor temporal registration.
Camera captured video content is prone to a lot of noise, particularly, when the lighting conditions are not ideal or the camera aperture/exposure settings cannot be intelligently adjusted. The noise in video influences the bit-rate and visual quality of video encoders and can significantly alter the effectiveness of video processing algorithms. Further, the noise leads to introduction of coding artifacts at a given bit-rate. This typically requires video processing algorithms to do a lot of fine-tuning in the presence of noise to be effective. Hence, de-noising is a key pre-processing operation in video or still image encoders. The key challenge in de-noising is in achieving the noise reduction while preserving the underlying spatiotemporal signal from artifacts such as spatial blur, motion blur, motion artifacts, and temporal flicker.
The image and video de-noising problems have been the target of active research for over two decades. From simple spatial averaging, the methods have evolved to include coring/shrinkage based methods in the wavelet domain and motion compensated temporal filtering. These techniques have a fairly high computational complexity because they involve a fairly large spatiotemporal support volume and require intelligent means of determining the weights needed to combine the pixels according to their similarity to the neighborhood of the pixel being de-noised. In addition, the several simple to complex noise estimation techniques have been studied with varying degrees of success to control the level of filtering according to the actual noise variance. Recent advances offer high quality de-noising at a fairly high computational complexity by increasing the spatiotemporal support and evaluating intelligent weights for combining these samples to remove noise while preserving the signal.
Other features of the present embodiments will be apparent from the accompanying drawings and from the detailed description that follows.